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A  Decision  Support  Model  for  Valuing  Proposed  Improvements 
in  Component  Reliability 

Presenter:  Dr.  Keebom  Kang  joined  the  Naval  Postgraduate  School  in  1988,  where  he 
teaches  supply  chain,  logistics  engineering  and  computer  simulation  modeling  courses  for  the 
MBA  program.  His  research  interests  are  in  the  areas  of  logistics  and  simulation  modeling  in 
various  military  applications.  He  received  his  Ph.D.  in  Industrial  Engineering  from  Purdue 
University.  Prior  to  joining  NPS,  he  was  on  the  faculty  of  the  Industrial  Engineering  Department 
at  the  University  of  Miami,  Coral  Gables,  Florida  (1983-1988).  He  had  held  visiting  professor 
positions  at  Syracuse  University  (Summer,  1985),  Georgia  Institute  of  Technology  (Fall,  2003), 
Asia  Institute  of  Technology  in  Thailand  (Winter,  2004),  and  Pohang  Institute  of  Science  and 
Technology  in  Korea  (Spring,  2004). 

Presenter:  Michael  Boudreau,  Colonel,  US  Army  (Ret),  has  been  a  senior  lecturer  at 
the  Naval  Postgraduate  School  since  1995.  While  an  active  duty  Army  Officer,  he  was  the 
Project  Manager,  Family  of  Medium  Tactical  Vehicles,  1992-1995.  He  commanded  the  Materiel 
Support  Center,  Korea,  1989-1991  and  the  Detroit  Arsenal  Tank  Plant,  1982-1984.  COL 
Boudreau  is  a  graduate  of  the  Industrial  College  of  the  Armed  Forces;  Defense  Systems 
Management  College;  Army  Command  and  General  Staff  College;  Long  Armour-Infantry 
Course,  Royal  Armoured  Corps  Centre,  United  Kingdom;  and  Ordnance  Officer  Basic  and 
Advanced  courses.  He  holds  a  Bachelor  of  Mechanical  Engineering  degree  and  Master’s  of 
Business  degree  from  Santa  Clara  University,  California. 

Presenter:  Uday  Apte,  is  Visiting  Professor  of  Operations  Management,  Graduate 
School  of  Business  and  Public  Policy,  Naval  Postgraduate  School,  Monterey,  CA,  and 
Associate  Professor,  Cox  School  of  Business,  Southern  Methodist  University,  Dallas,  TX.  He 
teaches  operations  management  courses  in  the  Executive  and  Full-time  MBA  programs.  His 
areas  of  expertise  and  research  interests  are  in  service  operations,  supply  chain  management 
and  globalization  of  information-intensive  services. 

Prior  to  joining  the  Cox  School,  he  worked  for  over  ten  years  in  managing  information 
technology  and  operations  functions  in  the  financial  services  and  utility  industries.  Since  then 
he  has  consulted  with  several  major  US  corporations  and  international  organizations  including 
IBM,  Texas  Instruments,  Nokia,  Kinko’s,  Nationwide  Insurance,  Nations  Bank  and  The  World 
Bank. 

He  holds  a  PhD  in  Decision  Sciences  from  the  Wharton  School,  University  of 
Pennsylvania,  where  he  taught  in  the  MBA  and  undergraduate  business  programs  for  over  ten 
years.  His  earlier  academic  background  includes  a  MBA  from  the  Asian  Institute  of 
Management,  Manila,  Philippines,  and  Bachelor  of  Technology  from  the  Indian  Institute  of 
Technology,  Bombay,  India. 

Dr.  Apte  has  published  over  30  articles,  five  of  which  have  won  awards  from  professional 
societies.  His  research  articles  have  been  published  in  prestigious  journals  including, 
Management  Science,  Journal  of  Operations  Management,  Decision  Sciences,  HE 
Transactions,  Interfaces,  and  MIS  Quarterly.  He  has  co-authored  one  book,  Manufacturing 
Automation  and  has  completed  work  on  another  co-authored  book,  Managing  in  the  Information 
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Abstract 


Developing  a  methodology  and  a  tool  for  estimating  the  operational  availability  (Ao)  of  a 
weapon  system  based  on  the  component-level  reliability  and  maintainability  data  is  the  goal  of 
this  research.  Specifically,  we  present  two  spreadsheet  models  and  one  discrete-event 
simulation  model  using  Arena  simulation  language.  The  first  two  models  support  lifecycle  cost 
calculations  and  are  static  in  nature.  The  third  model  incorporates  the  interactions  among 
reliability,  time  to  repair  and  operational  availability  into  a  discrete-event  simulation  model  that 
can  support  a  weapon-system-level  risk  analysis.  These  models  are  developed  as  proof-of- 
concept  to  demonstrate  the  potential  methodology  using  hypothetical,  yet  realistic  data. 

Introduction 

The  US  Department  of  Defense  is  engaged  in  a  number  of  management  initiatives 
(related  to  weapon  system  logistics  and  support)  intended  to  provide  reduced  lifecycle  cost 
while  simultaneously  improving  operational  availability,  Ao.  Performance-based  Logistics  (PBL) 
is  one  such  program  that  entails  the  establishment  of  a  particular  kind  of  contractual  vendor- 
client  relationship  between  a  logistic-service  provider  and  a  weapon-system  manager.  The 
Quadrennial  Defense  Review  mandated  the  DoD  implement  PBL  in  order  to,  “compress  the 
supply  chain  and  improve  readiness  for  major  weapons  systems  and  commodities”  (OSD,  2001, 
56).  A  key  aspect  of  PBL  contracts  is  their  outcome  focus;  the  client  organization  is  supposed 
to  specify  key  performance  goals,  and  allow  the  vendor  to  determine  the  best  way  of  obtaining 
those  goals  (ASN-RDA,  2003). 

This  paper  will  not  re-examine  the  core  questions  of  whether  PBL  works,  or  why  it  works, 
as  those  questions  have  been  examined  extensively  elsewhere  (e.g.,  Berkowitz,  et  al.,  2003). 
Rather,  we  take  as  our  starting  point  the  question  of  how  best  to  value  the  desired  outcomes  of 
a  PBL  contract.  After  all,  as  contractual  vehicles,  the  price  of  the  services  to  be  provided  must 
be  negotiated.  Also,  given  a  limited  budget  but  a  proactive  program  manager,  there  will  always 
be  more  opportunities  to  improve  logistical  support  for  a  weapon  system  than  dollars  available 
to  fund  those  opportunities. 

We  assume  that  opportunities  to  improve  logistics  outcomes  should  be  valued  on  the 
basis  of  the  cost-effectiveness  of  those  opportunities.1  As  in  the  private  sector,  the  cost 
effectiveness  of  an  opportunity  (investment)  is  its  mission-value-over-time  (profit,  in  the  case  of 
the  private  sector)  divided  by  its  cost-over-time.  It  would  thus  be  a  mistake  to  take  the  cost 
differentials  of  various  logistic  service  alternatives  as  a  statement  of  value  because  cost  in  no 
way  informs  the  value  of  that  service  to  the  weapon-system  operator.  Even  if  one  is  willing  to 
assume  that  current  expenditures  are  cost  effective  (and  hence,  any  cost  reduction  would  be 
even  more  cost  effective),  there  is  no  way  to  assess  one  alternative  against  another  without  a 
direct  measure  of  value;  mere  cost  differentials  ignore  the  fact  that  the  alternatives  may  have 
different  impacts  on  mission  value. 

We  will  further  assume  that  the  mission  value  of  a  logistical  service  is  a  function  of 
weapon-system  performance,  as  neither  a  weapon-system  component  (such  as  a  fuel  cell)  nor 


1  Caplice  &  Sheffi  (1994),  in  reviewing  a  panoply  of  logistics  metrics,  categorized  metrics  based  solely  on 
comparisons  of  inputs  (such  as  cost  comparisons)  as  utilization  metrics,  while  they  categorized  comparisons  of 
outputs  per  input  (such  as  what  we  are  calling  cost-effectiveness)  as  productivity  metrics.  They  made  the  point  that 
utilization  measures  are  usually  related  to  process  (as  opposed  to  performance)  management. 


r,.isuMurii  v  hmu>1 


ACQUISITION  RESEARCH:  THE  FOUNDATION  FOR  INNOVATION 


-  156- 


a  logistic  element  (such  as  spares  inventory)  can  contribute  to  mission  objectives  except 
through  the  weapon  system.  From  a  warfighter’s  viewpoint,  a  weapon  system  is  either  capable 
of  supporting  a  mission,  or  it  is  not.  While  a  fuel  cell  may  be  a  necessary  condition  for  the 
system  to  be  mission  capable,  it  is  not  a  sufficient  condition. 

Operational  availability  (Ao)  is  a  primary  metric  used  to  determine  the  probability  that  a 
weapon  system  will  be  capable  of  supporting  a  mission.  For  example,  in  an  aircraft  squadron, 
Ao  of  85%  implies  that  an  average  of  85%  of  the  aircraft  will  be  available  to  fly  in  support  of 
some  mission  objective.  Goals  are  often  stated  for  Ao  levels,  and  mission  planning  must  take 
Ao  into  account.  Moreover,  neither  a  war  fighter  nor  a  resource  manager  wanting  to  make 
contingency  plans  should  be  content  with  knowing  the  nominal  (target)  or  the  average  Ao  level. 
He  or  she  should  have  a  sense  of  the  distribution  of  Ao  around  the  target  levels:  the  probability 
that  Ao  will  fall  below  some  critical  level. 

It  is  also  possible  to  measure  Ao  for  fuel  cells,  as  well  as  aircraft;  an  improvement  in  Ao 
for  the  fuel  cell  will  provide  at  least  some  marginal  improvement  in  Ao  for  the  aircraft.  But,  this 
improvement  will  not  be  one-to-one;  large  improvements  in  fuel-cell  availability  may  yield  only 
trivial  improvements  in  aircraft  availability,  depending  not  only  on  the  failure  rate  of  the  fuel  cells, 
but  on  the  performance  and  availability  of  all  the  other  critical  components  of  the  aircraft. 
Likewise,  better  fuel-cell  availability  will  reduce  the  risk  that  a  particular  weapon  system  will  not 
be  operational  for  a  particular  mission,  but  the  magnitude  of  that  risk  reduction  depends  on  the 
probability  that  all  the  other  critical  components  of  the  aircraft  are  available. 

Hence,  the  value  of  an  improvement  of  component  logistics  can  only  be  understood  in 
terms  of  the  performance  of  all  the  other  critical  components  of  a  weapon  system.  Similarly,  the 
value  of  an  improvement  in  a  single  logistics  element  (such  as  spares  inventory)  can  only  be 
determined  in  conjunction  with  other  key  logistics  elements. 

The  modeling  approach  we  will  outline  in  this  paper  has  applicability  beyond  PBL.  It  is 
useful  in  understanding  the  value  of  component-level  logistic  services,  or  services  directed  at 
only  a  subset  of  logistic  elements  (inventory  only,  or  depot-level  repair  only).  However,  we 
contend  that  an  implementation  of  PBL  that  is  fully  consistent  with  the  original  intent  of 
performance-based  service  acquisition  must  use  an  approach  similar  to  the  one  we  outline, 
because  it  is  impossible  to  put  a  value  (and,  hence,  a  contract  price)  on  those  services  without 
such  an  approach. 

II.  LITERATURE  REVIEW 

While  we  are  arguing  for  an  assessment  of  value  that  will  provide  a  more  complete 
picture  of  the  cost  effectiveness  of  a  PBL  proposal  (by  providing  a  numerator  to  a  productivity 
ratio),  we  recognize  that  an  estimation  of  the  lifecycle  costs  of  such  proposals  is  far  from  trivial. 
Outsourced  logistic  services  for  weapon  systems  are  particularly  difficult  to  cost;  for  example, 
the  ongoing  contract  management  (transaction)  costs  can  be  substantial,  but  are  rarely 
measured  (Domberger,  Jensen  &  Stonecash,  2002). 

We  think  such  transaction  costs  are  particularly  important  in  light  of  a  recent 
Government  Accountability  Office  report  (GAO,  2004)  that  was  critical  of  systems-level  PBL 
contracts;  this  document  recommended  greater  emphasis  on  PBL  contracts  at  the  component 
level,  especially  for  commodity-type  components  (which,  according  to  the  GAO,  reflected 
“commercial  best  practices”).  PBL  contracts  on  commodities  would  be  especially  appealing 
because  vendors  providing  commodities  can  expect  to  enjoy  economies  of  scale  that  the  DoD 
could  not  experience  (as  vendors  would  be  able  to  offer  those  commodities  across  a  broad 
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population  of  users).  These  increased  economies  of  scale  would  reduce  the  price  of  such 
services.  Unfortunately,  of  course,  aside  from  domestic  transportation  and  depot-level  spares 
for  a  relatively  small  set  of  components  used  commonly  between  defense  and  industry,  the 
number  of  critical  components  (or  logistics  elements)  of  weapon  systems  that  can  be 
considered  commodities  is  relatively  small.  For  non-commodity  items,  a  key  economic 
consideration  in  out-sourcing  is  the  increase  in  transaction  costs  entailed  by  dealing  with  an 
outside  vendor  (Gufstafson,  et  al.,  1996).  Such  costs  increase  substantially  when  one  is 
offering  a  PBL  contract  at  the  component  level.  As  we  will  show,  aside  from  the  additional 
burden  of  contract  maintenance  for  many  small  contracts,  the  proper  valuation  and 
management  of  such  component-level  contracts  entails  the  development  of  a  comprehensive 
model  which  incorporates  key  performance  dimensions  of  all  critical  components. 

Perhaps  in  an  effort  to  reduce  such  transaction  costs,  or  perhaps  in  response  to  a 
complaint  that  PBL  involved  too  many  metrics,  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  (USD-ATL,  2004)  recently  issued  guidance  for  PBL  metrics.  While 
clearly  indicating  that  PBL  could  be  applied  at  the  subsystem  or  major  assembly  level,  the 
memo  listed  five  key  performance  criteria:  1)  weapon  system  operational  availability,  2)  weapon 
system  operational  reliability,  3)  weapon  system  cost  per  usage,  4)  logistics  footprint  for  a 
weapon  system,  and  5)  response  time  required  for  weapon  system  logistics  support. 

Of  course,  these  measures  are  interrelated.  We  think  the  central  non-cost  measure  is 
operational  availability.  The  other  three  non-cost  measures  can  all  be  seen  in  some  ways  as 
subsidiary  to  availability.  Reliability  (e.g.,  time  to  failure),  footprint  (e.g.,  number  of  spares  and 
size  of  fielded  or  intermediate  maintenance  and  repair  facility)  and  response  time  (e.g.,  time  to 
repair)  are  all  critical  determinants  of  availability.  Yet,  there  may  be  good  reasons  to  measure 
reliability,  footprint  and  response  time  separately.  For  example,  reliability  affects  not  only 
availability,  but  also  the  probability  of  system  failure  in  the  field;  likewise,  footprint  affects  not 
only  availability,  but  operational  agility  as  well.  However,  operational  availability  in  many  ways 
summarizes  reliability,  response  time  and  footprint.  We  will  develop  a  model  in  the  next  section 
that  demonstrates  the  precise  interaction  between  time  to  failure,  time  to  repair,  and  spare 
inventory  levels.  It  also  demonstrates  how  these  variables  determine  availability.  Thus,  as  they 
affect  Ao,  footprint,  response  time,  and  even  reliability  are  all  process  and  not  performance 
measures.  We  will  focus  on  availability  (with  the  caveat  that  it  may  not  be  the  sole  determinant 
of  value)  because  it  is  necessary  to  an  understanding  of  value. 

In  specifying  performance  outcomes  (but  not  processes)  to  a  vendor,  PBL  contracts  are 
deliberately  designed  to  transfer  some  degree  of  operational  and  financial  risk  to  a  vendor 
(Doerr,  Lewis  &  Eaton,  forthcoming).  As  risk  transfer  is  an  intended  outcome  of  the  initiative, 
and  as  the  risk  of  falling  below  a  certain  level  of  operational  availability  is  an  important 
performance  dimension,  it  is  clearly  important  to  incorporate  the  risk  associated  with  operational 
availability  at  the  system  level  into  a  measure  of  value.  From  the  warfighter’s  point  of  view,  this 
risk  may  be  the  key  performance  dimension  (Eaton,  Doerr  &  Lewis,  forthcoming).  The 
warfighter,  after  all,  is  less  concerned  with  the  average  number  of  mission-capable  aircraft  than 
he  is  concerned  with  the  probability  that  he  will  have  enough  aircraft  to  fly  a  particular  mission. 
The  procedure  we  will  outline  allows  the  assessment  of  a  proposed  logistics  improvement  not 
only  on  the  average  impact  that  improvement  would  have  on  the  operational  availability  of  the 
aircraft,  but  on  the  risk  associated  with  the  operational  availability  of  the  aircraft  as  well. 

Weapon  systems  are,  of  course,  the  military’s  key  capital  assets  related  to  operational 
capacity,  and  the  logistics  services  in  question  can  be  seen  as  primarily  affecting  the  level  of 
operational  capacity  available  to  the  warfighter.  The  sort  of  risk  measurement  we  are  proposing 
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is  increasingly  recognized  as  central  to  the  valuation  of  operational  capacity  of  corporate  assets 
in  the  private  sector  as  well. 

Assessments  of  risk/return  profiles  for  capital  assets  are,  of  course,  behind  the  recent 
work  on  Real  Options  (Mun,  200x).  And  in  capacity  planning  in  particular,  the  incorporation  of 
risk  into  capacity  models  was  listed  in  a  recent  literature  review  as  a  key  area  in  which  research 
was  expected  to  develop  (Van  Miegham,  2003).  Risk-based  models  have  recently  been  applied 
to  the  acquisition  of  production  capacity  for  airfoils  used  in  military  aircraft  (Prueitt  &  Park, 

2003).  Mostly,  risk-based  capacity  models  deal  with  technological,  demand,  or  price 
uncertainty,  and  are  not  directly  applicable  to  the  valuation  of  logistic  services  and  the  uncertain 
impact  those  services  will  have  on  system  availability  (capacity).  The  point  we  are  making  is 
that  there  is  growing  consensus  that  a  proper  valuation  of  capacity-related  planning  (such  as  the 
planning  associated  with  offering  a  PBL  contract)  must  include  an  assessment  of  risk. 

In  this  paper  we  develop  three  models  as  decision-support  systems  (Keen  &  Morton, 
1978;  Power,  2002;  Turban  &  Aronson  1998).  The  term  “decision-support  system”  implies  use 
of  computer-based  systems  to: 

1 .  assist  the  warfighters  in  their  decision  process  in  semi-structured  tasks, 

2.  support,  rather  than  replace,  the  warfighter’s  judgment,  and 

3.  improve  the  effectiveness  of  the  practical  decision-making  process. 

The  dramatic  improvements  in  computer  power  and  software  capability  (such  as 
spreadsheet  and  simulation  models)  allow  convenient  access  to  powerful  decision-support 
systems  for  improved  decision  making.  Making  such  models  available  as  decision-support 
systems  is  the  primary  goal  of  this  research. 

III.  MODELS 

In  this  section,  we  present  two  spreadsheet  models  and  one  discrete-event  simulation 
model  using  Arena  simulation  language  (Kelton,  2004).  The  first  model  primarily  supports 
lifecycle  cost  calculations  but  ignores  the  interactions  among  reliability,  time  to  repair,  and 
operational  availability.  The  second  model,  while  it  does  address  these  basic  interactions,  does 
not  consider  the  full  range  of  lifecycle  costs.  However,  both  the  first  and  the  second  model  are 
static — they  can  only  support  average  case  analyses  and  sensitivity  analyses.  The  third  model 
incorporates  the  interactions  among  reliability,  time  to  repair  and  operational  availability  into  a 
simulation  model  that  can  support  a  risk  analysis,  but  which  does  not  directly  address  lifecycle 
cost  issues. 

In  their  current  form,  these  models  are  intended  as  a  proof-of-concept  only.  That  is,  we 
are  not  presenting  a  research  case  involving  field  data;  rather,  we  are  demonstrating  the 
potential  of  an  approach  using  hypothetical  data. 

3.1.  Spreadsheet  Lifecycle  Cost  Model  (Model  1) 

Model  1  is  a  compressive  lifecycle  cost  analysis  model  for  a  hypothetical  UAV 
(unmanned  aerial  vehicle)  case  study  intended  as  a  proof-of-concept  for  our  modeling 
approaches.  This  case  study  was  adapted  from  Logistics  Engineering  class  lecture  notes  at  the 
Naval  Postgraduate  School  (Kang,  2004).  The  complete  case  study  is  described  in  Appendix  A, 
and  the  spreadsheet  model  is  available  from  http://web.nps.navy.mil/ 

~mn4310/UAV  Model  t.xls. 

This  model  computes  the  total  system  lifecycle  cost  for  major  weapon  systems  from 
R&D  to  deployment  to  phase-out.  The  lifecycle  cost  includes  research,  development,  test  and 
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evaluation,  acquisition,  production,  operations  and  maintenance,  and  phase-out  costs.  This 
model  is  a  comprehensive  decision-support  tool  for  program  managers.  The  model  can  be  used 
to  establish  the  baseline  total  ownership  cost  of  major  weapon  systems  during  the  planning,  as 
well  as  operations,  stages.  The  user  can  conduct  sensitivity  analyses  on  various  input 
parameters  such  as  reliability,  manning,  training,  and  R&D  cost.  As  the  user  changes  any  of  the 
parameters,  the  model  immediately  updates  the  total  lifecycle  cost,  so  the  user  can  see  the 
financial  impact  of  input  parameter  changes  in  the  long  run.  We  suggest  the  reader  download 
the  spreadsheet  model  and  change  some  of  the  parameters  in  the  “INPUT”  worksheet. 


3.2.  Revised  Spreadsheet  Model  (Model  2)  and  Simulation  Model  (Model  3) 

A  shortcoming  of  the  spreadsheet  model  (Model  1)  is  that  it  cannot  analyze  the  dynamic 
relationship  between  reliability  and  operational  availability.  For  example,  deterioration  in 
reliability  of  a  certain  component  will  decrease  the  system’s  operational  availability.  At  the  same 
time,  the  workload  at  a  repair  shop  will  increase,  forcing  the  repair  turnaround  time  to  become 
longer,  which  in  turn  will  decrease  the  operational  availability  of  the  system.  In  Model  1,  the 
average  repair  turnaround  time  remains  the  same  regardless  of  the  changes  in  component 
reliability. 

To  overcome  this  limitation,  we  have  developed  a  discrete-event  simulation  model 
(Model  3)  that  can  be  used  along  with  a  revised  spreadsheet  model  (Model  2).  Model  2  is 
essentially  derived  from  Model  1 .  It  is  a  small-scale  spreadsheet  model  to  focus  on  reliability 
and  maintainability.  Given  logistics  input  parameters  (see  Figure  1),  Model  2  computes  spare- 
parts  requirements,  inventory,  transportation  and  repair  costs  followed  by  the  total  maintenance 
costs  over  the  lifecycle  of  the  system.  Model  2  does  not  consider  R&D  cost  or  infrastructure 
costs.  It  only  considers  variable  costs  while  operating  the  weapon  system.  Figure  2  shows  the 
total  lifecycle  maintenance  cost  of  $442,656,976  based  on  the  input  parameters  in  Figure  1.  To 
demonstrate  how  Model  2  could  be  used,  suppose  we  improve  the  MTBF  of  the  main  display 
unit  from  1,500  hours  to  2,000  hours.  The  total  cost  will  then  be  decreased  to  $440,319,492, 
representing  approximately  $2.3  million  savings  in  maintenance  cost.  This  is  valuable 
information  for  the  program  manager  when  s/he  makes  the  component-reliability  improvement 
decisions. 

Figure  1.  Input  Parameter  for  Model  2 


No  of  Squadrons 

4 

No  of  UAV  systems  per  squadron 

10 

No  of  Air  Vehicles  per  system 

4 

No  of  Ground  Control  stations  per  system 

2 

Ground  Equip  Monthly  Op  Hrs  Hours 

300 

hrs 

AV  Flying  Hours/Vehicle/month 

120 

hrs 

AutoLand  &  Launch/RecMonthly  Op  Hours 

60 

hrs 

Repair  Turnaround  Time 

10 

days 

Protection  Level  for  Critical  Components 

0.95 

Protection  Level  for  non-Critical  Components 

0.85 

Hourly  charge  for  repair  including  material  cost 

$500 

Transportation  cost  per  failure 

$200 

Annual  Inventory  rate 

21% 

Capital  Discount  rate 

10% 

Lifecycle 

20 

years 
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Ground  Control  Station  Components 

MTBF 

Unit  Cost 

Main  Display  Unit 

1000 

0.00100 

$  500,000 

Critical 

Power  Supply 

4000 

0.00025 

$  400,000 

Critical 

Power  Gen 

3500 

0.00029 

$  300,000 

Critical 

Air  Conditioner 

6000 

0.00017 

$  400,000 

Critical 

Guidance  &  Control 

500 

0.00200 

$  400,000 

NonCritical 

Other  Ground  Equip 

MTBF 

Unit  Cost 

Launch  &  Recovery  System 

500 

0.00200 

$  1,200,000 

Critical 

AutoLand  System 

1000 

0.00100 

$  2,000,000 

NonCritical 

Data  Terminal 

3000 

0.00033 

$  1,000,000 

NonCritical 

AV 

MTBF 

Unit  Cost 

Navigation/Avionics 

1000 

0.00100 

$  200,000 

Critical 

Engine 

500 

0.00200 

$  100,000 

Critical 

Propeller 

500 

0.00200 

$  50,000 

Critical 

Video  Scanner 

2500 

0.00040 

$  150,000 

NonCritical 

IR  Scanner 

450 

0.00222 

$  150,000 

NonCritical 

IR  Data-Link 

800 

0.00125 

$  200,000 

NonCritical 

Figure  2.  Sample  Output  of  Model  2 


Annual  Spare  Inventory  Cost 

$  2,688,000 

per  squadron 

Annual  Repair  Cost 

$  8,800,857 

per  squadron 

Annual  Transportation  cost 

$  328,034 

per  squadron 

Total  cost  per  squadron  per  year 

$  11,816,891 

Total  Annual  cost 

$  47,267,566 

Total  Lifecycle  Cost 

$442,656,976 

Once  the  cost  analysis  is  completed  (using  Model  2),  the  same  input  parameters  are 
used  for  the  simulation  model  (Model  3)  to  estimate  the  operational  availability  and  other 
performance  measures  of  the  system  (e.g.,  probability  that  the  operational  availability  falls 
below  some  critical  level).  Model  2  and  Model  3  (simulation  model)  complement  each  other. 

3.3.  Simulation  Scenarios 

In  this  simulation  model  (Model  3),  we  only  consider  the  critical  components  (engine, 
propeller,  avionics  computer)  for  a  squadron  of  10  UAV  systems  with  40  air  vehicles  (see 
Appendix  A).  When  one  of  these  critical  components  fails,  the  faulty  component  is  removed 
from  the  air  vehicle,  and  an  RFI  (ready-for-issue)  spare  is  installed.  The  faulty  component  is 
sent  to  the  repair  shop  to  be  fixed.  After  repair,  it  becomes  an  RFI  spare.  When  a  critical 
component  fails,  and  an  RFI  spare  is  not  available,  the  air  vehicle  will  be  grounded  (and  will 
become  not  mission  capable,  or  NMC)  until  an  RFI  component  is  available.  A  failure  of  non- 
critical  components  may  degrade  readiness,  but  the  system  is  assumed  to  be  operable  (that  is, 
mission  capable  or  MC). 

The  input  parameters — such  as  MTBF  and  number  of  spares  for  each  component,  repair 
times  (in  hours),  transportation  delay  (one  way,  in  days) — are  read  from  the  spreadsheet  (see 
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Figure  3).  When  a  component  fails  in  Scenario  1 ,  it  requires  9  days  (4.5  days  one  way)  of 
transportation  delay  with  10  hours  of  repair  work;  this  work  follows  a  triangular  distribution  with  a 
mode  of  10  hours,  an  upper  limit  of  50%  above  the  mode  (i.e.,  15  hours)  and  a  lower  limit  of 
50%  below  the  mode  (i.e.,  5  hours).  The  waiting  time  at  the  repair  shop,  if  any,  is  estimated 
inside  the  simulation.  The  repair  turnaround  (TAT)  time  of  10  hours  in  Figure  1  for  Model  2 
approximates  the  repair  TAT  of  Scenario  1. 


Figure  3.  Simulation  Input  Spreadsheet 


Scenario 

MTBF_ 

Eng 

MTBF_ 

Prop 

MTBF_ 

AvComp 

Spare 

Engines 

Spare 

Props 

Spare 

AvComps 

Eng 

Repair 

hrs 

Prop 

Repair 

hrs 

AvComp 

Repair 

hrs 

Trans 

Delays 

(Days) 

1 

1000 

500 

500 

4 

6 

6 

10 

10 

10 

4.5 

2 

1250 

500 

500 

4 

6 

6 

10 

10 

10 

4.5 

3 

1500 

500 

500 

4 

6 

6 

10 

10 

10 

4.5 

4 

1000 

750 

500 

4 

6 

6 

10 

10 

10 

4.5 

5 

1000 

1000 

500 

4 

6 

6 

10 

10 

10 

4.5 

6 

1500 

1000 

500 

4 

6 

6 

10 

10 

10 

4.5 

7 

1000 

500 

500 

10 

10 

10 

10 

10 

10 

4.5 

8 

1000 

500 

500 

4 

6 

6 

10 

10 

10 

2.25 

9 

1000 

500 

500 

4 

6 

6 

10 

10 

10 

1 

10 

1500 

1000 

500 

4 

6 

6 

10 

10 

10 

1 

Given  the  input  parameters  in  Figure  3,  Model  3  simulates  each  scenario  over  1,000,000 
hours.  Multiple  scenarios  can  be  executed  in  one  simulation  run  (e.g.,  10  in  this  case).  The 
results  captured  for  each  scenario  are  the  average  operational  availability  (Ao)  for  the  air 
vehicles  in  the  squadron,  along  with  the  cumulative  distribution  of  operational  availability.  These 
results  are  tabulated  in  Figure  4._  The  cumulative  distribution  of  operational  availability  is  also 
depicted  graphically  in  Figure  5. 


Figure  4.  Simulation  Output:  Cumulative  Operational  Availability  and  the  Average 


Operational  Availability  for  Each  Scenario 


Cumulative  Operational 

Availability 

Avg 

Op 

Av 

Scenario  10%  20%  30%  40%  50%  60%  70%  80%  90%  100% 

1 

0.00 

0.00 

0.00 

0.00 

0.00 

0.02 

2.27 

32.25 

90.91 

100.00 

0.837 

2 

0.00 

0.00 

0.00 

0.00 

0.00 

0.01 

1.68 

28.76 

88.99 

100.00 

0.843 

3 

0.00 

0.00 

0.00 

0.00 

0.00 

0.01 

1.50 

27.53 

88.47 

100.00 

0.845 

4 

0.00 

0.00 

0.00 

0.00 

0.00 

0.01 

0.95 

20.50 

84.35 

100.00 

0.857 

5 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.55 

16.50 

80.87 

100.00 

0.865 

6 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.22 

11.70 

75.25 

100.00 

0.876 

7 

0.00 

0.00 

0.00 

0.00 

0.00 

0.04 

2.27 

32.43 

91.39 

100.00 

0.837 

8 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.03 

3.14 

52.74 

100.00 

0.906 

9 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.08 

15.37 

100.00 

0.948 

10 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.02 

6.63 

100.00 

0.962 
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Figure  5.  Cumulative  Operational  Availability 


Cumulative  Operational  Availability 


- Scenario  1 

- Scenario  2 

- Scenario  3 

- Scenario  4 

- Scenario  5 

- Scenario  6 

- Scenario  7 

- Scenario  8 

- Scenario  9 

- Scenario  10 

Let’s  assume  that  the  commander’s  goal  is  to  maintain  an  average  Ao  of  85%.  He  also 
knows  his  mission  capability  will  be  critically  jeopardized  if  Ao  falls  below  80%.  Therefore,  he 
wants  to  estimate  the  probability  that  this  event  might  happen,  maintaining  the  average  Ao  to  be 
above  85%.  The  results  in  Figure  4  show  that  the  average  Ao  of  Scenario  1  is  83.7%  (the  last 
column  of  Scenario  1)  and  the  probability  of  Ao  falling  to  80%  or  below  is  32.25%  (the  9th 
column  with  a  heading  of  80%  for  Scenario  1 ).  Scenario  1  is  not  acceptable  to  the  commander 
since  the  average  Ao  is  below  his  goal,  and  the  probability  of  Ao  falling  below  80%  seems  to  be 
too  high.  He  can  generate  more  scenarios  (e.g.,  Scenarios  2  through  10)  to  assess  the  impact 
of  changes  in  component  reliability  or  logistics  elements  (spare  parts,  repair  and  transportation 
times)  on  the  entire  system-level  Ao. 

In  Scenarios  2  and  3,  the  MTBF  of  an  engine  is  increased  from  1,000  hours  to  1,250  and 
1 ,500,  respectively.  In  Scenarios  4  and  5,  the  MTBF  of  a  propeller  is  improved  from  500  hours 
to  750  and  1,000,  respectively.  Improvement  in  Ao  can  be  observed  from  the  far  right-hand  side 
column  of  the  Figure  4.  Changes  in  Scenarios  4  and  5  are  preferred  to  those  of  Scenarios  2 
and  3.  In  Scenario  6,  the  MTBFs  of  both  the  engine  and  propeller  are  increased  respectively  to 
1 ,500  and  1 ,000.  The  overall  Ao  is  increased  to  87.6%  (from  83.7%  of  Scenario  1 ),  and  the 
probability  of  Ao  falling  below  80%  has  substantially  reduced  to  1 1 .7%  (from  32.25%).  Increase 
in  spare  parts  (Scenario  7)  does  not  improve  the  performance  at  all.  However,  significant 
reduction  in  transportation  time  (Scenario  8)  improves  the  system  performance.  In  Scenarios  8 
and  9,  when  the  transportation  delays  are  reduced  from  4.5  days  to  2.25  and  1  respectively,  Ao 
jumps  to  90.6%  and  94.8%,  respectively;  likewise,  the  probabilities  of  Ao  falling  below  80%  drop 
to  3.14%  and  0.08%,  respectively.  The  Scenario  10  is  the  same  as  Scenario  9  except  that  the 
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MTBFs  of  an  engine  and  a  propeller  are  increased  to  1 ,500  and  1 ,000,  respectively.  Ao  hits 
96.2%  with  the  probability  of  Ao  falling  below  80%  now  negligible  (0.02%). 

The  parameters  in  Scenarios  2  through  10  can  be  input  to  Model  2  to  compute  the  total 
maintenance  cost  for  each  scenario.  For  example,  by  entering  the  parameters  from  Scenario 
10  into  Model  2  in  Figure  2,  a  PM  will  note  results  in  a  total  lifecycle  maintenance  cost  of 
$375,712,781  (i.e.,  savings  of  approximately  $120  million  over  the  base  case  of  Scenario  1). 
Scenario  10  provides  an  Ao  12.5%  higher  than  Scenario  1  (from  83.7%  to  96.25%)  with  the  risk 
of  Ao  falling  below  80%  becoming  a  non-issue. 

Models  2  and  3  can  potentially  serve  as  a  communication  tool  between  the  budget 
community  and  warfighters.  When  reliability  improvements  are  made  on  several  components  in 
a  complex  system,  the  warfighter’s  primary  concern  is  readiness,  or  Ao,  while  the  budget 
analysts’  focus  is  on  financial  implications.  These  two  models  provide  valuable  solutions  to  both 
communities. 

IV.  SUMMARY 

Providing  reduced  lifecycle  cost  and,  at  the  same  time,  improving  operational  availability 
are  fundamental  goals  of  the  Performance-based  Logistics  (PBL)  and  other  logistics  initiatives 
of  the  U.S.  Department  of  Defense.  In  many  PBL  contracts,  the  contractual  arrangements  are 
typically  stipulated  at  the  level  of  individual  components  (such  as  a  fuel  cell)  or  a  logistic 
element  (such  as  inventory  of  certain  spare  parts).  While  achieving  component-level 
performance  goals  is  certainly  important,  what  really  matters  to  a  warfighter  is  the  operational 
availability  of  the  weapon  system.  Hence,  there  is  a  need  to  develop  a  methodology  and  an 
apparatus  for  estimating  the  operational  availability  (Ao)  of  a  weapon  system  based  on  the 
component-level  reliability  and  maintainability  data.  This  current  research  is  aimed  at  this  need. 

Specifically,  we  present  two  spreadsheet  models  and  one  discrete-event  simulation 
model  using  Arena  simulation  language.  The  first  model  primarily  supports  lifecycle  cost 
calculations,  but  ignores  the  interactions  among  reliability,  time  to  repair,  and  operational 
availability.  The  second  model,  while  it  does  address  these  basic  interactions,  does  not 
consider  the  full  range  of  lifecycle  costs.  However,  both  the  first  and  the  second  model  are 
static — they  can  only  support  average  case  analyses  and  sensitivity  analyses.  The  third  model 
incorporates  the  interactions  among  reliability,  time  to  repair  and  operational  availability  into  a 
simulation  model  that  can  support  a  weapon-system-level  risk  analysis.  In  their  current  form, 
these  models  are  developed  as  a  proof-of-concept.  That  is,  we  are  not  presenting  a  research 
case  involving  field  data,  but  rather  are  demonstrating  the  potential  methodology  and  a  tool 
using  hypothetical,  yet  realistic,  data. 
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APPENDIX  A: 

Unmanned  Aerial  Vehicle  (UAV)  Case  Study 

A  UAV  system  consists  of  four  air  vehicles  (AV's),  two  ground-control  stations  (GCSs), 
modular  mission  payloads  (MMPs),  data  links,  remote  data  terminals  (RDTs)  and  an  automatic 
landing  system.  A  total  of  8  squadrons  (two  squadrons  in  each  coast  of  CONUS,  and  one  each 
for  Pacific,  Indian,  Mediterranean,  and  Atlantic  Oceans)  will  be  established  to  accommodate  the 
new  system.  Each  squadron  will  have  its  own  intermediate-level  maintenance  capabilities. 

Each  squadron  will  have  10  VTUAV  systems.  Detachment  personnel  (for  each  UAV  system) 
will  consist  of  three  officers  (one  OIC  and  two  mission  officers),  three  Chief  Petty  Officers 
(CPOs)  and  12  enlisted.  1-Level  Maintenance  personnel  will  consist  of  one  officer,  one  Chief 
petty  officer  and  ten  enlisted.  Squadron  headquarters  personnel  will  be  made  up  of  seven 
officers,  ten  CPOs  and  twenty  enlisted.  Composite  costs  for  personnel  are  estimated  as  follows: 
Officer — $140,000  per  year,  CPO — $115,000  per  year,  Enlisted — $70,000  per  year. 

Production  begins  in  Fiscal  Year  2004,  with  all  VTUAV's  scheduled  for  field  tesing  in  the 
year  following  their  production.  A  total  of  80  VTUAV  systems  will  be  produced;  the  life-cycle  of 
the  program  is  estimated  to  be  30  years  (2005-2034).  The  risk  of  loss  of  an  AV  in  peace  time  is 
2-7%  per  year,  while  the  risk  of  loss  of  an  AV  in  operation  during  a  contingency  is  15-30%  per 
year.  A  chance  of  a  contingency  during  the  life-cycle  of  the  program  is  15%  per  year.  Lost  AVs 
will  be  replaced  the  next  year.  However,  no  orders  for  replacement  AVs  will  be  placed  last  5 
years  of  the  life-cycle  (i.e. ,  YR  2025  -  2029).  We  are  assuming  by  then  new  UAV  systems  will 
gradually  replace  the  current  ones. 

Research  and  development  costs  are  $15  million  for  FY  01,  $20  million  in  FY  02  and  $50 
million  in  FYs  03  and  04.  The  marginal  production  cost  of  AV  (with  payload)  is  $1  million.  The 
cost  of  maintaining  a  production  capability  throughout  the  life  of  the  system  is  $12  M  per  year  for 
every  year  any  aerial  vehicles  are  produced.  Thus,  the  annual  production  cost  of  AV  is  $12M  + 
$1M  *  (#  of  AV  produced).  Ground-Control  Equipment,  which  consists  of  two  GCSs,  RDTs,  test 
equipment  and  an  automatic  landing  system,  will  cost  $20  million  per  system.  The  l-level 
operating  cost  is  $6  million/yr  per  l-level  plus  an  additional  one-time  capital  investment  of  $25 
million  (including  installation  of  test  equipment)  prior  to  the  year  of  operation.  A  capital  discount 
rate  of  10%/yr  the  inflation  rate  of  4%/yr  will  be  used. 

Billet  requirements  are  based  on  all  personnel  fully  qualified/current/certified  to  perform 
all  missions/Navy  Enlisted  Classification  Code  (NEC)/Military  Occupational  Specialty  (MOS). 
Operators  are  required  to  have  functional  applications  of  the  use  and  control  of  the  UAV,  and 
will  be  trained  in  operation  of  all  aspects  of  the  UAV  navigation,  launch  flight  control  and 
recovery.  Officers  and  CPOs  will  attend  additional  training  on  preflight  planning,  mission  profile 
construction  and  UAV  tactical-intelligence  integration.  Costs  for  the  training  will  be 
$  1,600/person/week  for  the  basic  training  and  $3,000/  person/week  for  the  advanced  training. 

An  attrition  rate  of  25%  per  year  is  used  after  the  first  year,  including  personnel  rotation. 
Required  training  is  as  follows: 

Detachment  personnel 
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Basic  UAV  Training  (Officers,  CPOs,  junior  enlisted):  10  weeks 
Advance  Training  (Officers  and  CPOs  only):  5  weeks 

1-Level  Maintenance  personnel 

Basic  Maintenance  Training  (Officers,  CPOs,  junior  enlisted):  20  weeks 
Advance  Maintenance  Training  (Officers  and  CPOs  only):  5  weeks 

Squadron  Headquarters  personnel 

Basic  UAV  intelligence  course  (Officers,  CPOs,  junior  enlisted):  10  weeks 
Advance  Training  (Officers  and  CPOs  only):  5  weeks 

Spare  parts  management  will  be  consolidated  at  the  1-Level  on  a  one-for-one  exchange. 
We  will  assume  that  the  transportation  cost  is  $100  per  shipment  (i.e.,  $  200  per  failure). 

Spares  replacement  and  repair  materials  cost  will  be  equal  to  50%  of  the  value  of  spares  per 
year.  Sparing  levels  will  be  as  follows:  critical  units — 95%  and  non-critical  units —  85%. 
Maintenance  turnaround  time  (TAT),  including  transportation  delays,  for  1-Level  is  10  days  and 
D-Level  is  40  days.  It  is  assumed  that  80%  of  failures  can  be  repaired  at  the  1-Level  (thus  20% 
at  the  D-level).  Spare-level  calculations  are  based  on  "t  =10  (0.8)  +  40  (0.2)  =  16  days."  D- 
Level  cost  is  estimated  to  be  $5,000  per  repair  including  the  transportation  costs.  Ground 
equipment  is  expected  to  operate  300  hours  per  month;  the  AV  flying  hour  is  estimated  at  120 
hours  per  month  per  vehicle.  The  launch/recovery  and  the  auto-landing  systems  are  used  20% 
of  the  time  the  ground-control  station  is  in  operation  (i.e.,  60  hours  per  month).  POL  (petroleum, 
oil  and  lubricant)  costs  are  estimated  at  $60  per  flight  hour.  The  MTBF  of  each  component,  its 
cost,  and  the  required  protection  level  (customer  service  level)  are  included  as  follows: 


MTBF 

Cost 

Criticality 

Ground  Station  (2  per  VTUAV  svstem) 

Main  Display  Unit 

1,500  hrs 

$ 

500,000 

critical 

Power  Supply 

4,000  hrs 

$ 

400,000 

critical 

Power  Generator 

3,500  hrs 

$ 

300,000 

critical 

Air  Conditioner 

6,000  hrs 

$ 

400,000 

critical 

Guidance  &  Control 

500  hrs 

$ 

400,000 

non-critical 

II.  Other  Ground  Equipment  (1  per  VTUAV  system) 


Launch/Recovery  System 

500  hrs 

$1,200,000 

critical 

Auto-landing  System 

1,000  hrs 

$2,000,000 

non-critical 

Data  Terminal 

3,000  hrs 

$1,000,000 

non-critical 

III.  AV  and  Pavload 

Engine 

500  hrs 

$  100,000 

critical 
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Propeller 

500  hrs 

$ 

50,000 

critical 

Navigation/avionics 

1,000  hrs 

$ 

200,000 

critical 

Video  Scanner 

2,500  hrs 

$ 

150,000 

non-critical 

IR  Scanner 

450  hrs 

$ 

150,000 

non-critical 

IR  Data-Link 

800  hrs 

$ 

200,000 

non-critical 

The  System  activation/deactivation  plan  is  as  follows: 


System  Activation  plan: 

(2  squadrons  at  a  time) 


FY  2005  -  20  systems 
FY  2006  -  20  systems 
FY  2007  -  20  systems 
FY  2008  -  20  systems 


System  Deactivation: 
(phase-out)  plan 
(2  squadrons  at  a  time) 


FY  2031  -  20  systems 
FY  2032  -  20  systems 
FY  2033  -  20  systems 
FY  2034  -  20  systems 
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2003  -  2006  Sponsored  Acquisition  Research  Topics 

Acquisition  Management 

■  Software  Requirements  for  OA 

■  Managing  Services  Supply  Chain 

■  Acquiring  Combat  Capability  via  Public-Private  Partnerships  (PPPs) 

■  Knowledge  Value  Added  (KVA)  +  Real  Options  (RO)  Applied  to  Shipyard 
Planning  Processes 

■  Portfolio  Optimization  via  KVA  +  RO 

■  MOSA  Contracting  Implications 

■  Strategy  for  Defense  Acquisition  Research 

■  Spiral  Development 

■  BCA:  Contractor  vs.  Organic  Growth 
Contract  Management 

■  USAF  IT  Commodity  Council 

■  Contractors  in  21st  Century  Combat  Zone 

■  Joint  Contingency  Contracting 

■  Navy  Contract  Writing  Guide 

■  Commodity  Sourcing  Strategies 

■  Past  Performance  in  Source  Selection 

■  USMC  Contingency  Contracting 

■  Transforming  DoD  Contract  Closeout 

■  Model  for  Optimizing  Contingency  Contracting  Planning  and  Execution 
Financial  Management 

■  PPPs  and  Government  Financing 

■  Energy  Saving  Contracts/DoD  Mobile  Assets 

■  Capital  Budgeting  for  DoD 

■  Financing  DoD  Budget  via  PPPs 

■  ROI  of  Information  Warfare  Systems 

■  Acquisitions  via  leasing:  MPS  case 

■  Special  Termination  Liability  in  MDAPs 
Logistics  Management 

■  R-TOC  Aegis  Microwave  Power  Tubes 
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■  Privatization-NOSL/NAWCI 

■  Army  LOG  MOD 

-  PBL  (4) 

■  Contractors  Supporting  Military  Operations 

•  RFID  (4) 

■  Strategic  Sourcing 

■  ASDS  Product  Support  Analysis 

■  Analysis  of  LAV  Depot  Maintenance 

■  Diffusion/Variability  on  Vendor  Performance  Evaluation 

■  Optimizing  CIWS  Life  Cycle  Support  (LCS) 

Program  Management 

■  Building  Collaborative  Capacity 

■  Knowledge,  Responsibilities  and  Decision  Rights  in  MDAPs 

■  KVA  Applied  to  Aegis  and  SSDS 

■  Business  Process  Reengineering  (BPR)  for  LCS  Mission  Module 
Acquisition 

■  Terminating  Your  Own  Program 

■  Collaborative  IT  Tools  Leveraging  Competence 

A  complete  listing  and  electronic  copies  of  published  research  within  the  Acquisition 
Research  Program  are  available  on  our  website:  www.acquisitionresearch.org 
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